Hessian Matrices via Automatic Differentiation

We investigate the computation of Hessian matrices via Automatic Differentiation, using a graph model and an algebraic model. The graph model reveals the inherent symmetries involved in calculating the Hessian. The algebraic model, based on Griewank and Walther’s state transformations [7], synthesizes the calculation of the Hessian as a formula. These dual points of view, graphical and algebraic, lead to a new framework for Hessian computation. This is illustrated by giving a new correctness proof for Griewank and Walther’s reverse Hessian algorithm [7, p. 157] and by developing edge pushing, a new truly reverse Hessian computation algorithm that fully exploits the Hessian’s symmetry. Computational experiments compare the performance of edge pushing on sixteen functions from the CUTE collection [1] against two algorithms available as drivers of the software ADOL-C [4, 8, 14], and the results are very promising.